In the World Wide Web (“WWW”) environment, client machines effect transactions to Web servers using the Hyptertext Transfer Protocol (“HTTP”), which is a known application protocol providing users access to files (e.g., graphics, images, etc.) using a standard page description language known as Hypertext Markup Language (“HTML”). In the WWW paradigm, a network path to a server is identified by a Uniform Resource Locator (“URL”) having a special syntax for defining a network connection. Use of an HTML-compatible browser (e.g., Safari) at a client machine involves specification of a link via the URL. In response, the client makes a request to the server identified in the link and, in return, receives a document from the server or other object formatted according to HTML.
Many websites need to enforce some kind of policy regarding client requests for security purposes. The most commonly used policy by almost every internet service provider is URL filtering policy. The URL filtering policy allows users to access only the set of web resources that they have authorization to access.
At many popular Web sites, the capacity demand is much greater than can be served by one server. Accordingly a switch device or a gateway security device may be employed to distribute requests across a pool of servers or on to the network.
FIG. 1 is a diagram of a switch module engaged in a conventional policy enforcement such as URL filtering service behavior. A client device 105 sends a HTTP request, for example, to a switch device 110. Switch device 110 can also have load balancer functionalities to better perform policy enforcement, e.g., AX Series Server Load Balancers manufactured by A10 Networks. However, it should be noted that switch device 110 is not necessarily limited to a load balancer and can be any type of gateway security device. A conventional switch device 110 will forward the entire request to an external device, e.g., proxy servers 172, which will then authenticate the client and the URL by looking information up in URL database 174. External devices can also comprise transparent proxy servers, non-transparent proxy servers, proxy appliances, and combinations thereof. If the client is allowed to access the URL, then the connection will typically be allowed and the traffic from the client will be forwarded through network 150 to destination servers 115. Similarly, if the connection is allowed, then destination servers 115 are allowed to communicate via network 150 to client device 105 as well.
Typically, a “session” is established between the client 105 and servers 115, wherein the session represents a set of connection-less transactions between the client and the destination server. For example, if the destination server is a credit card company Web site, the session involves a set of queries to the server from the client 105, together with the responses served from the servers 115.
FIG. 2 is a diagram of a conventional switch device successfully connecting a client device to a destination server based on authorization received from an external device. Once the request from client 105 is authorized by external device, e.g., proxy servers 172, as discussed above, the switch device can establish a connection between the client 105 and the destination servers 115 through network 150.
FIG. 3 is a diagram of a conventional switch device denying a client device request to a destination server based on a denial received from an external device. If the URL requested by client device 105 is not authorized by external device, e.g., proxy servers 172, then the connection is denied by switch device 110 and the client device is not connected to external servers 115. This is typically referred to as a “403 response case.”
FIG. 4 is a diagram of a conventional switch device denying a client request to a destination server as a result of not receiving a response from an external device in the requisite timeframe. In some cases, the external device, e.g., proxy servers 172 may not respond to a client request in time. Accordingly, the switch device 110 will flag a time-out condition and deny the request to client 105.
FIG. 5 is a diagram of a conventional switch device failing to connect to an external device. For example, the external device may be down and not responding. In this case, the switch device 110 fails to connect to external device, e.g., proxy servers 172. Accordingly, the client 105 fails to connect to destination servers 115.
FIG. 6 is a diagram of a conventional switch device not being able to process incoming requests because a request rate limit has been reached. Client devices 605 and 606 are limited in the number of requests that can be transmitted to the switch device 610 and processed by the external devices, e.g. proxy servers 672 in a given period of time. This feature ascertains that the processing capability of the proxy servers is not overburdened. If the client devices cross over that threshold, then a request rate limit is reached and the requests are denied.
FIG. 7 is a diagram illustrating how a connection match is created between the connection from the client device to the switch device and the connection from the switch device to the external device. As discussed before, a session 744 is initiated when a request is sent from client device 105 to switch device 110. Information regarding the initiated session is stored in a session table. When the switch device 110 forwards the request to the external device, e.g., proxy servers 172, another session 748 is created between the client device 110 and the external device. Information regarding this session is also stored in the session table. The two sessions are correlated with each other in the session table by including pointers to each other in the respective data structures. Accordingly, the original session ID can be matched with the external service session ID.
Typically, URL-based policies are always enforced for web traffic. For large organizations such as Internet Service Providers (“ISPs”), the policy enforcement is complicated and is usually based on either the client IP address or user information. The policy enforcement thus requires a significant amount of processing power. Because the gateway device 110 in conventional network topologies does not do any of the policy processing, it needs to be done in an external device, e.g., proxy servers 172.
As a result, in conventional deployment scenarios, the gateway device 110 forwards an entire copy of the original request to the external devices, e.g., servers 172 and enforces the policy based on the response from the external devices. However, most of the content in the requests forwarded over by the gateway device are not used for policy processing in the external device. This results in inefficiencies within the network. First, parsing through all the requests from client devices to filter out the necessary content puts an undue processing burden on the external device. Second, transmitting the entire request, most of which will not be used by the external device creates occupies needless bandwidth in the network. As a result many external devices are required to perform this additional processing at added costs.